Numerical Analysis

Numerical analysis is concerned with the accurate and eecient evaluation of mathematical expressions, especially on computers with oating point arithmetic. While scientists have always been concerned to some extent with numerical computation, the modern discipline of numerical analysis is almost entirely a product of the period since 1950 during which there has been an explosion in the availability of electronic computers. There are three main issues: To organize computations so that there is minimum accumulation of error in oating point arithmetic; To organize computations eeciently, so that they consume the least possible resources; and To obtain accurate numerical approximations to quantities which may not have explicit mathematical expressions. In other words, do it accurately, do it quickly, do it cheaply. It might be thought that numerical evaluation consists largely of translating textbook formulae into a computer programming language, a mere coding exercise, but this is very far from the case. Direct translations of expressions from mathematical theory are seldom optimal, and very often are found to fail in circumstances when a less obvious numerical process would have succeeded. The focus on numerical results means that one is not limited to direct expressions, but can evaluate functions which are deened only indirectly, for example through integrals, diierential equations, series or as solutions to equations. An important example is the maximum likelihood estimator for a nonlinear statistical model. Indeed, an indirect method is often preferred for numerical evaluation even when a direct expression exists. While eeciency and accuracy are both aims, it is accuracy which takes precedence, since a slightly slower accurate program is invariably preferred to a faster one with unreliable accuracy. Errors arise from three sources: (i) errors in the input data, (ii) computation errors due to nite precision arithmetic and (iii) approximation error. The rst of these are not under the control of the calculation; in fact it might be considered to be the special concern of the statistician. Computation error appears because of the diierence between exact arithmetic and the nite-length arithmetic available on digital computers and hand calculators. Approximation error occurs when the computed expression is not exactly equal to the theoretical quantity even in exact arithmetic. An integral is replaced with a sum, for example, or an innnite series is evaluated only to a nite number of terms. EEciency is usually measured by counting basic oating point operations ((ops), such as additions, subtractions, multiplications and divisions. …